On the Geometry of Maximum Entropy Problems
نویسندگان
چکیده
We show that a simple geometric result suffices to derive the form of the optimal solution in a large class of finiteand infinite-dimensional maximum entropy problems concerning probability distributions, spectral densities, and covariance matrices. These include Burg’s spectral estimation method and Dempster’s covariance completion, as well as various recent generalizations of the above. We then apply this orthogonality principle to the new problem of completing a block-circulant covariance matrix when an a priori estimate is available.
منابع مشابه
3d Tomographic Reconstruction of Signnaltering Functions by the Maximum Entropy Method
Maximum Entropy Method (MEM) for the reconstruction of sign{ altering functions from two dimensional tomographic measurement data is developed. Three-dimensional algorithm for parallel beam geometry are considered. Results of numerical simulations for composite model are presented. keywords: Tomography, Maximum Entropy Method, Ill-posed problems
متن کاملEntropy, Information Theory, Information Geometry and Bayesian Inference in Data, Signal and Image Processing and Inverse Problems
The main content of this review article is first to review the main inference tools using Bayes rule, the maximum entropy principle (MEP), information theory, relative entropy and the Kullback–Leibler (KL) divergence, Fisher information and its corresponding geometries. For each of these tools, the precise context of their use is described. The second part of the paper is focused on the ways th...
متن کاملMaximum Entropy Density Estimation with Generalized Regularization and an Application to Species Distribution Modeling
We present a unified and complete account of maximum entropy density estimation subject to constraints represented by convex potential functions or, alternatively, by convex regularization. We provide fully general performance guarantees and an algorithm with a complete convergence proof. As special cases, we easily derive performance guarantees for many known regularization types, including `1...
متن کاملMinimization Problems Based on a Parametric Family of Relative Entropies I: Forward Projection
Minimization problems with respect to a one-parameter family of generalized relative entropies are studied. These relative entropies, which we term relative α-entropies (denoted Iα), arise as redundancies under mismatched compression when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative entropies are a generalization of the usual r...
متن کاملA Note on the Bivariate Maximum Entropy Modeling
Let X=(X1 ,X2 ) be a continuous random vector. Under the assumption that the marginal distributions of X1 and X2 are given, we develop models for vector X when there is partial information about the dependence structure between X1 and X2. The models which are obtained based on well-known Principle of Maximum Entropy are called the maximum entropy (ME) mo...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- SIAM Review
دوره 55 شماره
صفحات -
تاریخ انتشار 2013